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Test scores provide one method of assessing student achievement, but a vari- 
ety of factors affect student performance. The Office of Educational Quality 
and Accountability (EQA) was created by the state Legislature in July 2000 to 
examine many of these additional factors by conducting independent audits of 


schools and districts in Massachusetts. The agency uses these audits to: 
| Provide a comprehensive evaluation of each school district's performance; 
-) Publish annual reports on selected district's performance; 


1 Monitor public education performance statewide to inform policy decisions; 


and 


1 Provide the public with information that helps the state hold districts and 


schools, including charter schools, accountable. 


On March 7-22, 2005, the EQA conducted an independent examination of 
Boston Public Schools for the period of 2001-2004. This school district was 
selected for Tier |, Tier Il, and Tier Ill reviews. In the Tier | review, the EQA ana- 
lyzed Boston students’ performance on the Massachusetts Comprehensive 
Assessment System test and identified areas where students were performing 
significantly better or worse than the state average. In Tier Il, the EQA examined 
critical factors that affected student performance in five major areas: leadership, 
governance, and organization; curriculum and instruction; assessment and eval- 
uation; academic support services; and business and finance. The Tier Ill review 
was school- and classroom-based and was intended to determine the extent to 
which the district's plans and programs reached the classroom level and 
addressed the students’ needs. (For more on the EQA’s three-tier examination 


process, see Appendix A.) 


The review was based on source documents supplied by the Massachusetts 
Department of Education and Boston Public Schools; correspondence sent 
prior to the EQA team’s on-site visit; interviews with a representative from the 
school committee, the district leadership team, and school administrators; and 
additional documents submitted while the EQA team visited the district. The 
report does not take into account documents, revised data, or comments that 
may have been supplied after June 2004. However, district leaders were invit- 


ed to provide more current information. 


EDUCATIONAL MANAGEMENT AUDIT COUNCIL ACTION 


After reviewing this report, the Educational Management Audit Council unanimously 


accepted its findings at its meeting on September 16, 2005. 
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Putting the Data in Perspective | 


Boston, MA 


“Mie 
DISTRICT 


Population: 589,141 

Median family income: $44,151 
Largest sources of employment: 
education, health care, social services, 
financial services, insurance 

Local government: Mayoral] 

City Council 


SCHOOLS AND STUDENTS 


School committee: 7 members 
Number of schools: 136 
Student enrollment: 
Total: 57,742 
White: 14 percent 
Hispanic: 31.2 percent 
African-American: 45.5 percent 
Asian-American: 8.9 percent 
Limited English proficiency: 
17 percent 
Low income: 73.5 percent 
Special education: 19.6 percent 


Sources: 2000 U.S. Census and 


Massachusetts Department of Education. 


MCAS Performance at a Glance, 2004 


DTS PR Ga STATE 


Average Proficiency Index 63 78 
English Language Arts 

Proficiency Index 70 84 
Math Proficiency Index 56 7) 


Performance Rating 


® 


a 
Very High Moderate Low Very Critically 
High Low Low 


The Proficiency Index is another way to look at MCAS 
scores. It is a weighted average of student performance 
that shows whether students have attained or are making 
progress toward proficiency, which means they have met 
the state's standards. A score of 100 indicates that all stu- 
dents are proficient. 


HOW DID STUDENTS PERFORM? 
Massachusetts Comprehensive 
Assessment System (MCAS) 

Test Results 


Students in grades 3-8 and Grade 10 are required to 
take the MCAS tests each year in one or more spec- 
ified subject areas, including English language arts 
(ELA), math, and science and technology/engineer- 
ing (STE). Beginning with the class of 2003, stu- 
dents must pass the Grade 10 math and ELA tests 
to graduate. Those who do not pass on the first try 
may retake the tests several more times. 


The EQA analyzed current state and district MCAS 
results to determine how well district students as a 
whole and subgroups of students performed com- 
pared to students throughout the commonwealth. 
The EQA analysis sought to answer the following 
five questions: 


1. Are all eligible students participating in all required programs and assessments? 


| Overall, Boston students did not participate in the ELA and math assessments at 


required levels; however, eligible students did participate in STE assessments at 


required levels. 


. In Boston, the 93.3 percent student participation rate on the 2004 MCAS ELA sub- 


test was 1./ percentage points lower than the state’s 95 percent requirement. 


-|_ In Boston, the 93.9 percent student participation rate on the 2004 MCAS math sub- 


test was 1.1 percentage points lower than the state’s 95 percent requirement. 


. In Boston, the 95.7 percent student participation rate on the 2004 MCAS STE sub- 


test was 0.7 percentage points higher than the state’s 95 percent requirement. 
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BOSTON SCORES COMPARED TO STATE AVERAGES 


Percentage of students at each proficiency level on MCAS 


Above Standard 


Below Standard 


-100 


English Language Arts 


Science & Technology/ 
Engineering 


WM Advanced 

@) Proficient 

_ Needs Improvement 
Wi Warning/Failing 


38.0 
49.0 


State Boston State 


Boston State Boston 


2. Are the district’s students reaching proficiency levels on the MCAS examination? 


1 Thirty-two percent of Boston’s students scored ‘Proficient’ or higher on the 2004 
MCAS tests, which is lower than student performance rates across the state. The 


state average was 53 percent. 


1 Asignificantly larger percentage of students scored ‘Proficient’ or higher on ELA 
(41 percent) than on math (27 percent) in 2004. 


By the winter 2004/05 MCAS retest, 80 percent of the Boston class of 2005 
passed the Grade 10 math and ELA tests, compared to 94 percent statewide. 


1 By the same retest, 74 percent of the Boston class of 2006 passed both tests, 
compared to 90 percent statewide. 
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BOSTON ELA SCORES COMPARED TO MATH SCORES 


Percentage of students at each proficrency level on MCAS 


Above Standard 


Below Standard 


100 


100 


English Language Arts Math 


M8 Advanced 
Proficient 


- Warning/Failing 
MM Needs Improvement 


274 24 227 


2001 2002 2003 2004 2001 2002 2003 2004 


3. Has the district's MCAS test performance improved over time? 


Between 2001 and 2004: 


1 The district maintained a ‘Low’ performance rating, but it improved its 


Proficiency Index scores by 23 percent during the review period. 


_| The percentage of Boston students attaining proficiency on the math sub- 
tests was 41.8 percent higher in 2004 than in 2001. 


4. Do MCAS test results vary among subgroups of students? 


1 Less than one-third of both African-American and Hispanic students in 
Boston attained proficiency on the 2004 MCAS tests. Both of these propor- 


tions were 58.6 percent lower than that of White students. 


. Less than one-third of both students eligible for free or reduced-cost lunch 
(FRL/Y) and students eligible for free lunch (FL) in Boston attained proficien- 
cy on the 2004 MCAS tests. These proportions were 85.2 and 48.0 percent 


lower than that of FRL/N students in Boston. 
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: 5. Has the MCAS test performance of the district's student subgroups improved 
x over time? 
= Between 2002 and 2004, in ELA and math: 


HOW 


1 Boston’s subgroups improved moderately, consistent with all Boston stu- 
dents and students statewide. . 


| Students not eligible for free or reduced lunch (FRL/N) improved significant- 
ly, and the gap closed by 10.3 percentage points in ELA; for all Boston stu- 
dents, the gap closed by 5.4 points and by 3.5 points among all students 


statewide. 


Boston Public Schools, 2001-2004 


Performance at a Glance 


Ratings on Performance Indicators 


In this area, 


districts are rated on 18 performance 


indicators. Boston received the following ratings: 


13 


1 


Excellent Satisfactory Poor Unsatisfactory 


Areas of Strength 


Q 


District publications such as the annual Guide to the Boston 
Public Schools for Families and Students, a school choice 
newspaper distributed through parent information centers, 
and the district's website kept parents and community mem- 
bers informed about school options, policies, and MCAS test 


progress. 


District leaders could communicate instantly with faculty and 


staff through the MyBPS website. 


The number of non-ceritifed teachers decreased during the 


review period. 


Principals indicated that they were kept well informed of dis- 
trict policies through communication with triad leaders and 


regular, written updates. 


Areas for Improvement 


_] 


In 2004, the Boston Public Schools was identified on the 
state's ‘In Need of Improvement list for math, as mandated by 
the United States Department of Education. Fifty-eight of 


Boston's 136 schools appeared on that list. 


Sixty administrators, a substantial percentage of the total 
administrative workforce, were not certified, including a 
deputy superintendent, three headmasters, and three elemen- 


tary principals. 


The district experienced a high teacher turnover rate during 


the review period. 


The BPS’ Policy Manual had not been updated in 17 years. 


Wil Ate EANCTORS. DRIVE STUDENT PER- 
FORMANCE? 


Leadership, Governance and Organization 


Ultimately, the success or failure of district leadership was 
determined by how well all students performed. Boston’s per- 
formance rating was ‘Low,’ but it improved its Proficiency 


Index scores significantly between 2001 and 2004. 


The school district leadership consisted of the superintendent, 
the seven-member school committee, and the Mayoral and 
City Council, which were united in their “focus on children.” 
The school committee and the superintendent, who was hired 
in 1995, shared high expectations and worked to ensure that 
all of Boston’s students had access to quality educational 


opportunities. 


Planning 


Plans for improving student performance need to start with a 
clear vision and set of priorities backed by in-depth data to 
inform decision-making. Boston had a five-year improvement 
plan, Focus on Children Il. Covering 2001-2006, this provided 
a clear statement of expectations at the district level and spe- 
cific direction at the school level. Each school developed a 
Whole School Improvement Plan and created an Instructional 
Leadership Team (ILT) to review school progress three times 
per year. Focus on Children If! included the following Six 
Essentials for Whole School Improvement: 


1 Focus on literacy and math. 
. Use student work and data to identify student needs. 
| Improve instruction and assess progress. 


Focus on professional development for teachers and 


principals, to improve instruction. 
oi Identify and replicate best practices for instruction. 


1  -Engage families and community partners to support 


Whole School Improvement. 
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Governance 


The Boston Public Schools’ school level governance structure was complex. It was composed of a 
school-site council and a personnel subcommittee. The Whole School Improvement Plan was 
designed and written by the principal and the ILT; the ILT along with the School Site Council were 
responsible for monitoring the plan. Additionally, the personnel subcommittee was responsible for pro- 
viding input on the employment and deployment of teachers. 


Sixty administrators, a substantial percentage of the total administrative workforce, were not certified, 
including a deputy superintendent, three headmasters, and three elementary principals. However, the 
district's number of non-certified teachers decreased during the review period. The district experi- 


enced a high teacher turnover rate during the review period, 


For the last two years of the review, the district provided mentoring for only a fraction of the school 
year as specified in the contract. According to principal and teacher interviewees, the lack of a year- 


long mentoring program for all new teachers contributed to a higher rate of teacher turnover. 


In general, principals were not able to interview applicants from outside the school district until plac- 
ing applicants transferring from inside the district in open slots. According to principal interviewees, 
this practice often limited their ability to fill open positions in their schools with the best qualified can- 
didates. 


The Boston Public Schools’ Policy Manual had not been updated in 17 years. 


The Office of Research, Assessment, and Evaluation and the MyBPS website provided the district's 


schools and teachers with substantial resources for the analysis of student achievement data. 


In 2004, the Boston Public Schools was identified on the state’s ‘In Need of Improvement list for math, 
as mandated by the United States Department of Education. Furthermore, 58 of Boston’s 136 schools 


were also on that list. 


Communication 


District publications such as the Guide to the Boston Public Schools for Families and Students, an 
annual school choice newspaper distributed through parent information centers, and the district's 
website, kept parents and community members informed about school choices, school progress on 
the MCAS test and other tests, school achievements, school policy in each zone/triad, and district 
policies. 


Although principals did not rely on a_ district policy manual, they indicated that they were kept well 
informed and were well aware of district policies through communication with triad leaders and regu- 
lar written updates. District leaders could communicate instantly with all BPS faculty and staff through 
MyBPS. 


Boston Public Schoals, 2001-2004 


Performance at a Glance 


Ratings on Performance Indicators 


In this area, districts are rated on 32 performance indi- 
cators. Boston received the following ratings: 


Excellent Satisfactory Poor 


Unsatisfactory 


Curriculum 


Areas of Strength 


QO 


In Focus on Children II, the district developed a clear 
vision and corresponding expectations for improve- 


ment. 


Through the Whole School Improvement Plan process, 
there was a direct correlation between learner needs 


and professional development. 


Through the Collaborative Coaching and Learning 
model, teachers received support and direction in 


meeting a range of learner needs. 


Areas for Improvement 


o 


Educators of students with disabilities did not take 
part in school wide professional development during 


the review period. 
The achievement gap for subgroups remained wide. 


Many of the district's special education students were 
taught by special education teachers who had not 


received content-based professional development. 


VEC AC iA Cou on R IVER Sd UD ENT: PER - 
FORMANCE? 


Curriculum and Instruction 


The most recent district and school improvement plans in 
Boston consistently demonstrated high academic expectations 
for student achievement and effective instruction — an essen- 
tial element of efforts to improve student performance. The 
leadership team in Boston coordinated curriculum, instruction, 
and professional development, and established a direct rela- 
tionship among the implementation of instructional strategies, 
instructional time, and improved student achievement. These 
factors may have contributed to the district’s improvement over 


the review period. 


Aligned curricula and effective instruction 


In Focus on Children /!, the district developed a clear vision and corre- 
sponding expectations that improved student achievement in the 
aggregate; however, the implementation of the components for district- 
wide improved student achievement was incomplete. Successfully 
implemented throughout the district were relatively small classroom 
sizes, the scope and sequence of math instruction, and the adherence 


to the benchmarks in elementary ELA instruction. 


But, the district provided little effective oversight of curriculum imple- 
mentation, and had no requirement that ELA or math instructional or 
program directors had specific certifications or expertise in the content 
areas. The district did not provide teachers with the opportunity to par- 
ticipate in the development and periodic revision of curriculum. 
Through the Collaborative Coaching and Learning (CCL) model, 
teachers received support and direction in implementing Readers’ and 
Writers’ Workshops methodologies and customizing instruction to 
meet a range of learner needs. However, educators of students with 
disabilities did not take part in CCL and school-wide professional 
development during the review period. The district exceeded profes- 


sional development spending requirements. 


The ELA and math resources used and the quality of instruction were 


inconsistent. Advanced Work Classes provided noticeably higher qual- 
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ity instruction to students than the regular classrooms. In regular classrooms, the EQA found inconsistent 


requirements for students to use higher order thinking; some teachers limited instruction to basic skills. 


Also, the implementation of the necessary components to improve student achievement was more evident at 
the elementary level than at the middle and high school levels. Elementary classrooms largely displayed evi- 
dence of a district focus on student achievement. Although the quality of instruction varied, most classrooms 
utilized a variety of resources, such as ELA curriculum materials, books, technology, and math manipulatives 


to enhance direct instruction, independent learning, pair groupings, and cooperative groups. 


Further enhancements to the elementary learning environment included the use of interns, volunteers, and 
displays on classroom walls emphasizing ELA goals and student work space. Middle and high school class- 
rooms showed less evidence of a focus on improving student achievement. ELA classes were largely 
teacher-directed, with much less group work than in the elementary classrooms, less display of student work, 
fewer reading and math materials in classrooms, and less student access to computers, mostly located in 
separate laboratories. Computer hardware and software were available for the use of teachers and students 
in all buildings. However, the quantity, quality, and currency of equipment varied dramatically from site to site. 


Examiners found that computer technology was not an integral part of instruction across the district. 


The district's classroom assessment standards, practices, and expectations were linked to the state curricu- 
lum frameworks. While the district provided tools for using disaggregated assessment information that teach- 
ers increasingly applied in adjusting instruction, special education instructional programs remained substan- 


tially separate during the review period. 


The district made progress in inreasing access to Advanced Placement (AP) courses for all high school stu- 
dents and in assigning personnel to teach these courses. Individual schools that developed curriculum cal- 
endars or pacing charts, aligned with grade level curriculum, had more intra-school horizontal and vertical 
alignment of the curriculum in ELA and math and were more likely to complete more of the district's curricu- 
lum at each grade level. The district increased the number of students graduating in their senior year, and 
increased the percentage of students achieving Competency Determination. But, the achievement gap for 


subgroups remained wide. 


Policies and practices were in place to monitor staff attendance. However, the district did not analyze pat- 
terns, trends, and chronic absenteeism to determine the effect on student achievement and learning. The dis- 
trict had policies and procedures for student transition, transfers, dropouts, discipline, retention, suspension, 


and exclusion, and conducted regular data analysis by racial subgroup. 
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Boston Public Schools, 2001-2004 


Performance at a Glance 


Ratings on Performance Indicators 


In this area, districts are rated on 28 performance 
indicators. 


1 


Excellent Satisfactory Poor 


Unsatisfactory 


Boston received the following ratings: 


Areas of Strength 


= 


The district implemented MyBPS, a data-driven system 
for evaluating district programs and Services. 


Boston Public Schools used many benchmarks and 
formative, Summative, and standardized assesments In 
addition to the MCAS test. 


The district developed and distributed a school report 
card for all schools to parents and community mem- 


bers. 


Areas for Improvement 


E} 


Only 30 percent of reviewed teacher performance 
evaluations were completed in a timely manner; less 
than 5 percent referred to student performance. 


The district's attendance rate was below the state 
average and its chronic absenteeism rate was approx- 
imately 30 percent, 45 percent at the high school. 


The central office's ability to reallocate staff was 
severely restricted by Boston Teachers’ Union con- 
tracts. 


No districtwide policies for addressing poor stu- 
dent attendance. 


Disproportionately high suspension rates and 


low retention rates for Hispanic students. 


WHAT FACTORS DRIVE STUDENT PER- 
FORMANCE? 
Assessment and Evaluation 


Student assessment data include a wealth of information 
for district and school leaders on strengths and weakness- 
es in the local system, which provides valuable input on 
where they should target their efforts to improve achieve- 
ment. Boston Public Schools established the infrastructure 
for assessment and evaluation systems. Through its 
Office of Research, Assessment, and Evaluation and the 
development and implementation of the MCAS test data- 
base program, the district implemented MyBPS, a data- 
driven system for the evaluation of district programs and 


services and for resource allocation. 


Improving student achievement 


Boston Public Schools provided data teams in each school 
to assist the principal in data analysis and training other 
teachers in the use of MyBPS. In addition to the MCAS test, 
BPS used many benchmarks and formative, summative, 
and standardized assessments during the review period. 
The district developed and distributed a district report card 
of students’ educational progress and of teaching and 
learning programs through an evaluation of assessment 
data, for all schools, that was widely circulated to parents 


and community members . 


Although MyBPS was designed to make data collection easy 
and accessible, its implementation and use was not wide- 
spread among classroom teachers, according to building- 


level adminsitrators and teachers. 


Examiners observed the Collaborative Coaching and 
Learning Workshop Model in use across elementary grades; 
however, the use of this pedagogy declined as students pro- 
gressed to upper grade levels. Although the CCL model was 
used across all schools, the lack of consistency in ELA con- 


tent to be used at each grade level was noticeable from 


school to school. The use of the math curriculum was more structured 


across the district, training and the use of Investigations and the Connected 


Math Program (CMP) were inconsistent with students other than regular 


education. Implementation of differentiated instruction in order to engage all 
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learners was not generally observed. 


Budget and staffing decisions 


Although the central office and principals had the authority to reallocate finances, their ability to 
reallocate staffing was severely restricted by negotiated contracts with the Boston Teachers’ 
Union. The contract that the district negotiated with the teachers’ labor union outlined an instruc- 
tional remediation process that imposed rigorous deadlines on the evaluating administrator. All 
final unsatisfactory evaluations became the source of exhaustive procedural grievances. The 
contract language between the teachers’ labor union and the district severely restricted man- 
agement’s ability to assign the most capable and skilled staff members to the lowest perform- 


ing students. 


Staff evaluation 


The district did not conduct personnel evaluations consistently. The most recent evaluations for 
some teachers and administrators dated back to the mid-1990s. Principals had limited ability to 
allocate staff and resources and wide variations of both were observed from school to school. 
Although many of the teachers’ evaluations were well written, the district was only minimally 
meeting state law requiring evaluations of all professional-status teachers in alternating years. 
None of the reviewed teacher performance evaluations contained specific student test scores, 
despite language in the teachers’ collective bargaining contract that allowed evaluators to 
inform teachers of student test data informally. 


Student attendance 


Students who come to school every day have a better chance to learn and achieve. During the 
review period, the district had policies in place that required all students to attend school. 
However, the district’s attendance rate of 91.7 percent was below the state average of 94.2 per- 
cent and did not meet the state standard of 93 percent. Despite strict attendance policies, the 
district's chronic absenteeism rate was approximately 30 percent, and chronic absenteeism at 
the high school level averaged approximately 45 percent. The district policy that required retain- 
ing a student only once in elementary, middle, or high school conflicted with the district's atten- 


dance policy and resulted in retention rates that were lower than expectations. 


Boston Public Schools, 2007-2004 


_ Performance at a Glance 


Ratings on Performance Indicators 


In this area, districts are rated on 11 performance 
indicators. Boston received the following ratings: 


ie 


0 ) 


Excellent Satisfactory Poor — Unsatisfactory 


Areas of Strength 


Q 


The district generated annual grant-funding in excess 
of $65 million to develop and enhance academic sup- 


port programs. 


The district's effort to provide student academic sup- 
port programs contributed to moderate growth In stu- 
dent achievement results, by 7.7 percent in ELA and 8 


percent in math. 


Areas for Improvement 


rT 


Despite ELA and math improvement in the aggregate, 
the district performed below the state averages in 


both ELA and math. 


Supplemental student support services were not avail- 
able at all schools for all students during the review 


period. 


Wiig CG LOR Sse RI VEsod UDENT PER] 
FORMANCE? 


Academic Support Services 


Students who are at risk of failing or dropping out need 
additional support to ensure that they stay in school and 
achieve proficiency. 


Services 


The district's format for developing the Whole School 
(WSIP), 


Massachusetts Department of Education, delineated indi- 


Improvement Plan approved by _ the 
vidual school sites’ curriculum accommodations to provide 


academic support. 


The district's MyBPS program, implemented in 2003-04, 
allowed for instant access to student performance data to 
plan instruction and the alignment of programs and 


resources. 


As a result of the district's ability to generate annual grant 
funding in excess of $65 million during the review period, it 
was able to continue developing and enhancing academic 
support programs, despite reduced grant-funding for 
MCAS academic support services. 


In the aggregate, the district's efforts to provide student 
academic support programs contributed to student per- 
formance growth of 7.7 percent in ELA and 8 percent in 


math. 


Despite improvement in ELA and math in the aggregate, 
the district performed below the state average in both ELA 


and math throughout the review period. 


Supplemental student support services were not available at all schools 


for all students during the review period. The district's early intervention 


program did not successfully ensure that all Grade 3 students were read- 


ing at proficiency by the end of the academic year. 
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Although classroom observations and school-based interviews provided evidence that more 
formative assessments were being given and collected on a consistent basis, there was less 
consistency on the capacity of Instructional Leadership Teams to analyze this data and guide 


teachers in instructional decision-making based on the data collected. 


Although the district had a promotion policy linked to grade-level benchmarks, interviews with 
school-based personnel indicated that students who did not reach the benchmarks were pro- 


moted based on a record of prior retentions. 


Boston Public Schools offered a wide range of interventions that were school-based decisions 
and examples of school choice initiatives. Parents who were interviewed during the on-site visit 
offered different opinions as to whether families were granted assignments based on their choic- 
eS. 


Boston Public Schools, 2001-2004 


Performance at a Glance 


Ratings on Performance Indicators 


In this area, districts are rated on 28 performance indi- 
cators. 


0 2 ) 


Excellent Satisfactory Poor Unsatisfactory 
Boston received the following ratings: 


Areas of Strength 


1 Boston Public Schools were financially stable with 


generally good budgeting practices. 


| Grants and foundation funds were aligned with the 
district's Focus on Children Il and the Whole School 


Improvement Plan initiatives. 


1 The City of Boston was very financially supportive of 


the district. 


Areas for Improvement 


-i  Building-level administration did not consistently pro- 


vide input regarding capital needs. 


| Principals differing efforts to pursue grant funds 
resulted in inequities between schools that were 


acknowledged and accepted by the district. 


_| Principals and deputy superintendents said they had 
control of their budgets but limited control over per- 


sonnel. 
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Business and Finance 


Boston Public Schools was financially stable and had 
generally good budget practices. Overall, the district's 
budget documents were clear and informative, and its 
budget process was transparent and included the input 
of key stakeholders. 


Budget Process and Financial Support 


Between 2001-2004, Boston Public Schools received 
63.4 percent of its total educational funding from the 
City of Boston and other local sources. From FY2001 
to FY2004, BPS received $809 million in state Chapter 
70 Aid. The district's Focus on Children I/ provided the 
focus of budget development decisions at all levels 
and in all departments. Massachusetts Department of 
Education data showed that the city contributed more 
than the required local contribution of $1,404,190,287. 


Despite budget reductions in fiscal year 2003, the dis- 
trict continued implementing and advancing Focus on 
Children ff and Whole School Improvement initiatives. 
District and school leadership solicited and sustained 
extensive community and business partnerships to 
promote overall academic improvement and achieve- 
ment. The school district's budget document was thor- 
ough and explained the budget and the spending plan 


in extensive detail. 


The district employed a budget office staff that worked 
closely with building principals and other program 
directors. The City of Boston also had a position for 
day-to-day monitoring of school department transac- 


tions in its finance department. Reports were made 


available to managers each accounting period through the communi- 


ty accounting system. The district used a requisition and purchase 


order process for all transactions. The district had policies and proce- 


LE 
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dures in place that governed the expenditures and transfers of the district as well as of 


the individual sites and departments. 


Grants and foundation funds were aligned with the district's Focus on Children II and 
Whole School Improvement Plan initiatives. For the review period, the school district 
was cited for various grants management and reporting requirement issues in the City’s 
Single Audit, in accordance with OMB Circular A-133. Some of the issues recurred in 
subsequent fiscal years to varying degrees. Boston Public Schools did not dispute any 


of these findings, and implemented corrective action plans. 


The schools’ budgets were adequate at all levels until the 2002 and 2003 school years. 
Expense budgets for materials and supplies were reduced across all schools, resulting 
in inadequate resources, despite the fact that the city contributed more than the required 
local contribution. Principals and deputy superintendents stated that they had control of 
their budgets but had limited control over their personnel. Financial information was 
readily available and used efficiently by all administrators with budget authority. The 
availability of grant funds was effectively communicated to building and district admin- 
istrators. 


Grant management was conducted at the building level with assistance from the 
school’s development office. Principals acknowledged the need to seek out and obtain 
grant funds to enhance the educational programs in the buildings. These efforts result- 


ed in inequities between schools that were acknowledged and accepted by the district. 


Facilities 


The City invested significant funds each year under review for capital improvements to 
school facilities. Capital planning in the school district was a collaborative effort at the 
central administration level. The building-level administration did not consistently pro- 
vide input regarding capital needs. Building renovations and improvements were an 
ongoing issue that all levels of officials acknowledged as the City was investing in proj- 


ects to address these needs. 
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CONC EUSTON 


The EQA examination found that Boston Public Schools established an infrastructure to 
enhance many of the components necessary to improve student achievement, particular- 
ly at the elementary level. However, implementation of these components was incomplete 
at the middle and secondary levels, particularly in the use of student groupings, person- 


nel evaluations, special education, and professional development. 


Boston Public Schools failed to remove obstacles to ensuring hiring, support for, and 
retention of the highest quality staff. The district lacked year-long mentoring for new teach- 
ers, consistent evaluations, and certified administrators; the Boston Teachers’ Union’s 
procedural grievances with unsatisfactory evaluations exacerbated the latter issue. 
Additionally, the district’s rate of teacher turnover was high, and its policies prevented prin- 
cipals from filling open positions with the most qualified candidates. Furthermore, despite 
the Boston Public Schools’ strict attendance policies, student absenteeism remained a 
problem. 


During the review period, according to the MCAS examination results, Boston scored in 
the ‘Low’ category. Additionally, there was an equity gap among subgroups, between 
schools. However, the district's efforts resulted in improvements in student performance. 
These results were due to the district's clear educational leadership. The impact of stable, 
informed leadership, a dedicated school committee, and a governance structure insulat- 
ed from election politics set Boston apart from other major urban systems in 
Massachusetts. With more emphasis on implementation, particularly at the middle and 
secondary levels, Boston could improve student academic performance more rapidly. 
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EQA’s three-tier examination process provides successively deeper levels of informa- 
tion about student performance. All school districts receive a Tier | review annually, but 


they do not all receive the full three-tier review every year. 


Based on the Tier | results, Educational Management Audit Council (EMAC) policy, 
and random sampling, approximately 60 districts statewide received a more in-depth 
Tier Il review. Still other districts — those that do not meet certain performance crite- 
ria set by the state Department of Education — received an enhanced Tier Il or even 
more detailed Tier Ill review. 


Tier |: Data-Driven Assessment 


Annually, the DOE and EQA’s staff assess each public school district’s results on the 
Massachusetts Comprehensive Assessment System (MCAS) tests to find out how 


students are performing. The Tier | review seeks to answer five basic questions: 
1. Are the district’s students reaching proficient levels on MCAS? 


2. Do MCAS test results vary among subgroups of students (such as minority and 
low-income students and students with disabilities)? 


3. Has the districts MCAS test performance improved over time? 
4. Has the MCAS test performance of the district's student subgroups improved over 
time? 


5. Are all eligible students participating in all required programs and assessments? 


Tier Il: Standards-Based Examination 


Districts with MCAS results that fall within certain thresholds of performance, particu- 
larly districts that score below average, may be selected to receive a Tier Il review. 
This review seeks to provide a more complete picture of why the district is performing 
at that level, examining district management, planning and actions and how they are 
implemented at the building level. It focuses in particular on whether the district uses 


data to inform its efforts. 


The report uses 15 standards to analyze district performance in five major areas: leader- 


ship, governance, and organization; curriculum and instruction; assessment and eval- 
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uation; academic support services; and business and finance. EQA examines a total of 117 indi- 
cators to assess whether the district is meeting the standards and provides a rating for each indi- 


cator. 


Tier Ill: District Diagnostic (Fact-Finding) Review 


While the Tier Il review looks at how district policies are implemented at the building level, the 
Tier Ill review goes a level deeper and looks at what is happening in the classroom and how 
that affects school performance. In this review, EQA used qualitative data from observations 
of 448 classrooms in 48 schools (16 in each triad) and over 150 interviews were conducted to 
assess district practices expressed in the implementation of the 15 standards in the same five 
major areas as a Tier Il review. 


Boston Public Schools, 2001-2004 


Ks 
iw) 


TOO ESS OA Sih a OM 


CN Vile O Urea OMe el ST 2a 


ddV 


a 


V XIAN 


DoS ee i PRE Or Re Malan Gee 


EO SS YOO Wik” “sie eo @ns 


B 


APPENDIX 
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The vast majority of a school district’s funding is determined by the Chapter 70 program 
— the major program of state aid to public elementary and secondary schools. In addi- 
tion to providing state aid to support school operations, it also establishes minimum 
requirements for each municipality’s share of school costs. The following chart shows 
the amount of Boston Public Schools’ funding that came from the state and the amount 
that the town was required to contribute. Since 1996, both state funding and the 


required local contribution have increased significantly. 


WHERE DOES THE FUNDING FOR BOSTON PUBLIC SCHOOLS COME FROM? 


MM) Required Local 
Contribution 


Be State Chapter 70 
Aid 


1996 997 1998 1999 2000 2001 2002 2003 2004 


HOW IS THE FUNDING FOR BOSTON PUBLIC SCHOOLS ALLOCATED? 


L HR Mgmt. & Prof. Dev. 
$13,343,105 (2%) 


Leadership & Governance el 
$9,095,735 (1%) 


Total City Charges 


$203,250,149 (24%) Curriculum & Instruction 


$368,453,976 (42%) 
Business, Finance, & Other 
$230,062,407 (27%) 
Access, Opportunity, 
Student Support Services 
$35,232,097 (4%) 


Assessment & Evaluation 
$351,491 (0%) 
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